From Understanding to Controlling Privacy against Automatic Person Recognition in Social Media

Growth of the internet and social media has spurred the sharing and dissemination of personal data at large scale. At the same time, recent developments in computer vision has enabled unseen effectiveness and efficiency in automated recognition. It is clear that visual data contains private information that can be mined, yet the privacy implications of sharing such data have been less studied in computer vision community. This extended abstract presents a line of research that begins with the study of person identification in social media photos and progresses towards effective computer vision technology for anonymisation. 1. Understanding Privacy Figure 1. PIPA samples for identity X. Original (Day) split training and testing samples are shown in upper (left) and lower (right) halves. Person recognition in social media photos is a fledgling area in computer vision research; traditional focus has been face recognition and pedestrian re-identification, where vision algorithms have achieved impressive performances. Social media sets new challenges: non-frontal faces, severe occlusions, varying pose, etc. It is unclear how existing face recognition and re-identification techniques transfer to the social media setup. Relevant benchmarks and experimental setups have to be designed, and different methods need to be tried. This section reviews the first large-scale dataset and benchmark for person recognition in social media setup [4], and more realistic and challenging scenarios introduced by [1]. We describe the state of the art person recogniser [1]. Social media dataset: PIPA. The PIPA dataset [4] consists of social media photos on Flickr. It contains ∼40k images over ∼2k identities, and captures subjects appearing in diverse social groups and events. It is arguably the first large-scale social media dataset for person recognition. The first setup Zhang et al. [4] considered evaluates the recognition system on the samples of identities from the same contexts (clothing, event, etc.) as the training samples, referred to as the Original split. Oh et al. [1] has introduced more challenging scenarios according to the domain shift between the training and test samples (Album, Time, and Day). Figure 1 visualises the Original and Day splits.